Image processing system and method

ABSTRACT

An image processing system according to an embodiment includes a display unit capable of displaying stereoscopic images by displaying a group of parallax images and a display control unit. The display control unit displays an operation screen for receiving operations on medical image data on the display unit, displays selection information for selecting medical image data on the operation screen, and controls whether the selection information will be displayed as a stereoscopic image or a planar image depending on the content of medical image data selected by the selection information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2011-133164, filed on Jun. 15, 2011; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an image processingsystem and method.

BACKGROUND

In the past, apparatuses capable of generating 3-dimensional medicalimage data (hereinafter referred to as volume data) have beenpractically used as medical image diagnostic apparatuses such as anX-ray CT (computed tomography) apparatus, an MRI (magnetic resonanceimaging) apparatus, or an ultrasonic diagnostic apparatus. The medicalimage diagnostic apparatus collects imaging data by imaging a subjectand generates volume data by performing image processing on thecollected imaging data. Moreover, the medical image diagnostic apparatusgenerates a display image to be displayed on a monitor by performing avolume rendering process on the generated volume data. Here, the medicalimage diagnostic apparatus displays an operation screen for receivingoperations from an operator on a monitor in synchronization with imagingof a subject or generation or displaying of display images. Moreover,the medical image diagnostic apparatus performs imaging of a subject,generation or displaying of display images, and the like in accordancewith an operation received on the operation screen.

On the other hand, in recent years, stereoscopic monitors which enable2-parallax images imaged from two viewpoints to be perceivedstereoscopically using a dedicated device such as stereoscopic glasseshave been practically used. Moreover, stereoscopic monitors which enablemulti-parallax images (for example, 9-parallax images) imaged frommultiple viewpoints to be perceived stereoscopically with the naked eyesof a viewer by using a beam controller such as a lenticular lens havebeen practically used. The 2-parallax images or 9-parallax imagesdisplayed on the stereoscopic monitor are sometimes generated byestimating the depth information of images imaged from one viewpoint andperforming image processing on the images using the estimatedinformation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram for describing a configuration example of an imageprocessing system according to a first embodiment;

FIGS. 2A and 2B are diagrams for describing an example of a stereoscopicdisplay monitor that performs stereoscopic display using 2-parallaximages;

FIG. 3 is a diagram for describing an example of a stereoscopic displaymonitor that performs stereoscopic display using 9-parallax images;

FIG. 4 is a diagram for describing a configuration example of a medicalimage diagnostic apparatus according to the first embodiment;

FIG. 5 is a diagram for describing a configuration example of arendering processing unit illustrated in FIG. 4;

FIG. 6 is a diagram for describing an example of a volume renderingprocess according to the first embodiment;

FIG. 7 is a diagram for describing a display method according to thefirst embodiment;

FIG. 8 is a diagram for describing a display method according to thefirst embodiment;

FIG. 9 is a diagram for describing a configuration example of a storageunit and a control unit according to the first embodiment;

FIG. 10 is a diagram for describing generation of a thumbnail regionaccording to the first embodiment;

FIG. 11 is a diagram for describing generation of a thumbnail regionaccording to the first embodiment;

FIG. 12 is a diagram for describing an operation screen according to thefirst embodiment;

FIG. 13 is a flowchart illustrating the flow of a display controlprocess according to the first embodiment;

FIG. 14 is a flowchart illustrating the flow of a display controlprocess according to the first embodiment;

FIG. 15 is a diagram for describing an operation screen according to amodification example of the first embodiment;

FIG. 16 is a diagram for describing an operation screen according to amodification example of the first embodiment;

FIG. 17 is a diagram for describing a display example according to asecond embodiment; and

FIG. 18 is a diagram for describing a time schedule screen according toa third embodiment.

DETAILED DESCRIPTION

An image processing system according to an embodiment includes a displayunit capable of displaying stereoscopic images by displaying a group ofparallax images and a display control unit. The display control unitdisplays an operation screen for receiving operations on medical imagedata on the display unit, displays selection information for selectingmedical image data on the operation screen, and controls whether theselection information will be displayed as a stereoscopic image or aplanar image depending on the content of medical image data selected bythe selection information.

Hereinafter, embodiments of an image processing system and method willbe described in detail with reference to the accompanying drawings. Someterms used in the following embodiments will be described. “A group ofparallax images” is a group of images obtained by moving a viewpointposition at which an object is observed by a predetermined parallaxangle. For example, “a group of parallax images” can be generated byperforming a volume rendering process with respect to volume data withthe viewpoint position moved by a predetermined parallax angle.Moreover, for example, “a group of parallax images” can be generated byperforming a computation process so that a predetermined shape (forexample, a cuboid) can be stereoscopically perceived. That is, “a groupof parallax images” is made up of a plurality of “parallax images”having different “viewpoint positions.” Moreover, “parallax angle” is anangle determined by adjacent viewpoint positions among the respectiveviewpoint positions set to generate the “a group of parallax images” anda predetermined position (for example, the center of a space) within aspace represented by volume data. Moreover, “parallax number” is thenumber of “parallax images” necessary for the images to be perceivedstereoscopically on a stereoscopic monitor. Moreover, “9-parallax image”described below is “a group of parallax images” made up of nine“parallax images.” Moreover, “2-parallax image” described below is a“group of parallax images” made up of two “parallax images.”

First Embodiment

First, a configuration example of an image processing system accordingto the first embodiment will be described. FIG. 1 is a diagram fordescribing a configuration example of an image processing systemaccording to the first embodiment.

As illustrated in FIG. 1, an image processing system 1 according to thefirst embodiment includes a medical image diagnostic apparatus 110, animage archiving device 120, a workstation 130, and a terminal device140. The respective devices illustrated in FIG. 1 can communicatedirectly or indirectly with each other by a LAN (local area network) 2installed in a hospital, for example. For example, when a PACS (picturearchiving and communication system) is incorporated into the imageprocessing system 1, the respective devices transmit and receive medicalimages or the like to and from the other devices in accordance with theDICOM (Digital Imaging and Communications in Medicine) standard.

The image processing system 1 generates a group of parallax imagesserving as display images from volume data which is 3-dimensionalmedical image data collected by the medical image diagnostic apparatus110 and displays the group of parallax images on a stereoscopic monitorto thereby provide stereoscopic medical images to doctors and examinersworking in a hospital. Specifically, in the first embodiment, themedical image diagnostic apparatus 110 performs various image processeswith respect to volume data to generate a group of parallax images.Moreover, the medical image diagnostic apparatus 110, the workstation130, and the terminal device 140 include stereoscopic monitors thereof,and the group of parallax images generated by the medical imagediagnostic apparatus 110 is displayed on the monitors. Moreover, theimage archiving device 120 archives the volume data and the group ofparallax images generated by the medical image diagnostic apparatus 110.That is, the workstation 130 and the terminal device 140 acquire thegroup of parallax images from the image archiving device 120 and processthe group of parallax images and display the same on the monitors.Moreover, in the first embodiment, the medical image diagnosticapparatus 110, the workstation 130, and the terminal device 140 displayicons to be displayed on operation screens thereof so as to be perceivedstereoscopically. Here, “icons” are information for receiving operationsfrom an operator and are figures and characters, or a combinationthereof, designed to enhance visibility. Hereinafter, respective deviceswill be described in order.

The medical image diagnostic apparatus 110 is an X-ray diagnosticapparatus, an X-ray CT (computed tomography) apparatus, an MRI (magneticresonance imaging) apparatus, an ultrasonic diagnostic apparatus, aSPECT (single photon emission computed tomography) apparatus, a PET(positron emission computed tomography) apparatus, a SPECT-CT apparatusin which a SPECT apparatus and an X-ray CT apparatus are integrated, aPET-CT apparatus in which a PET apparatus and an X-ray CT apparatus areintegrated, or a cluster of these apparatuses. Moreover, the medicalimage diagnostic apparatus 110 according to the first embodiment cangenerate 3-dimensional medical image data (volume data).

Specifically, the medical image diagnostic apparatus 110 according tothe first embodiment generates volume data by imaging a subject. Forexample, the medical image diagnostic apparatus 110 collects imagingdata such as projection data or MR signals by imaging a subject andgenerates volume data by reconstructing medical image data of aplurality of axial planes along the body axis direction of the subjectfrom the collected imaging data. For example, the medical imagediagnostic apparatus 110 reconstructs medical image data of 500 axialplanes. A group of medical image data of the 500 axial planes is volumedata. The projection data or the MR (magnetic resonance) signalsthemselves of the subject imaged by the medical image diagnosticapparatus 110 may be referred to as volume data.

Moreover, the medical image diagnostic apparatus 110 according to thefirst embodiment performs various rendering processes with respect tothe generated volume data to generate a group of parallax images.

Moreover, the medical image diagnostic apparatus 110 according to thefirst embodiment includes a stereoscopic monitor (hereinafter, astereoscopic display monitor) as a display unit. The medical imagediagnostic apparatus 110 generates a group of parallax images anddisplays the generated group of parallax images on the stereoscopicdisplay monitor. As a result, the operator of the medical imagediagnostic apparatus 110 can perform an operation for generating thegroup of parallax images while checking stereoscopic medical imagesdisplayed on the stereoscopic display monitor.

Moreover, the medical image diagnostic apparatus 110 transmits thegenerated volume data and the generated group of parallax images to theimage archiving device 120. When transmitting the volume data and thegroup of parallax images to the image archiving device 120, the medicalimage diagnostic apparatus 110 transmits additional information.Examples of the additional information include a patient ID foridentifying a patient, an examination ID for identifying an examination,an apparatus ID for identifying the medical image diagnostic apparatus110, and a series ID for identifying each imaging by the medical imagediagnostic apparatus 110. Moreover, the additional informationtransmitted to the image archiving device 120 together with the group ofparallax images includes additional information regarding the group ofparallax images. Examples of the additional information regarding thegroup of parallax images include the number (for example, “9”) ofparallax image and a resolution (for example, “466×350 pixels”) of aparallax image.

The image archiving device 120 is a database in which medical images arearchived. Specifically, the image archiving device 120 according to thefirst embodiment stores the volume data and the group of parallax imagestransmitted from the medical image diagnostic apparatus 110 in a storageunit and archives the same. In the first embodiment, by using theworkstation 130 capable of archiving a large volume of images, theworkstation 130 and the image archiving device 120 illustrated in FIG. 1may be integrated with each other. That is, in the first embodiment, thevolume data and the group of parallax images may be stored in theworkstation 130 itself.

In the first embodiment, the volume data and the group of parallaximages archived in the image archiving device 120 are archived incorrelation with a patient ID, an examination ID, an apparatus ID, aseries ID, and the like. Thus, the workstation 130 and the terminaldevice 140 acquire necessary volume data and the necessary group ofparallax images from the image archiving device 120 by retrieving thesame using the patient ID, the examination ID, the apparatus ID, theseries ID, and the like.

The workstation 130 is an image processing apparatus that performs imageprocessing on medical images. Specifically, the workstation 130according to the first embodiment acquires the group of parallax imagesacquired from the image archiving device 120 and displays the acquiredgroup of parallax images on a stereoscopic display monitor. As a result,doctors and examiners who are viewers can view the stereoscopic medicalimages. In the first embodiment, although the medical image diagnosticapparatus 110 performs processes up to generation of the group ofparallax images, for example, the workstation 130 according to the firstembodiment may acquire volume data from the image archiving device 120,perform various rendering processes with respect to the acquired volumedata, and generate the group of parallax images.

The terminal device 140 is a device for allowing doctors and examinersworking in a hospital to view medical images. For example, the terminaldevice 140 is a PC (personal computer), a tablet PC, a PDA (personaldigital assistant), a mobile phone, or the like, operated by doctors andexaminers working in a hospital. Specifically, the terminal device 140according to the first embodiment includes a stereoscopic displaymonitor as a display unit. Moreover, the terminal device 140 acquires agroup of parallax images from the image archiving device 120 anddisplays the acquired group of parallax images on the stereoscopicdisplay monitor. As a result, doctors and examiners who are viewers canview the stereoscopic medical images.

Here, the stereoscopic display monitor included in the medical imagediagnostic apparatus 110, the workstation 130, and the terminal device140 will be described. Most popular general-purpose monitors areconfigured to display 2-dimensional images in a 2-dimensional plane andare unable to display 2-dimensional images stereoscopically. If a viewerwants to view images stereoscopically on a general-purpose monitor, anapparatus that outputs images to the general-purpose monitor may need todisplay 2-parallax images in parallel which can be perceivedstereoscopically by a viewer by a parallel viewing method or across-eyed viewing method. Alternatively, the apparatus that outputsimages to the general-purpose monitor may need to display images whichcan be perceived stereoscopically by a viewer by a complementary colormethod using glasses in which a red cellophane is attached to theleft-eye portion and a blue cellophane is attached to the right-eyeportion.

On the other hand, an example of the stereoscopic display monitor isknown, such as a stereoscopic display monitor which enables 2-parallaximages (also referred to as binocular parallax images) to be perceivedstereoscopically using dedicated devices such as stereoscopic glasses.

FIGS. 2A and 2B are diagrams for describing an example of a stereoscopicdisplay monitor which performs stereoscopic display using 2-parallaximages. The example illustrated in FIGS. 2A and 2B is a stereoscopicdisplay monitor that performs stereoscopic display by a shutter method,and shutter glasses are used as stereoscopic glasses worn by a viewerwho watches the monitor. The stereoscopic display monitor alternatelyoutputs 2-parallax images on the monitor. For example, the monitorillustrated in FIG. 2A alternately outputs a left-eye image and aright-eye image at a frequency of 120 Hz. As illustrated in FIG. 2A, aninfrared emitting unit is provided in the monitor, and the infraredemitting unit controls the emission of infrared light in accordance withan image switching timing.

Moreover, the infrared light emitted from the infrared emitting unit isreceived by an infrared receiving unit of the shutter glassesillustrated in FIG. 2A. A shutter is included in each of the left andright frames of the shutter glasses, and the shutter glasses alternatelyswitch between a light-transmitting state and a light-blocking state ofthe left and right shutters in accordance with the timing at whichinfrared light is received by the infrared receiving unit. Hereinafter,a process of switching between the light-transmitting state and thelight-blocking state of the shutter will be described.

As illustrated in FIG. 2B, each shutter includes an incident-sidepolarizer and an emitting-side polarizer, and further includes a liquidcrystal layer between the incident-side polarizer and the emitting-sidepolarizer. Moreover, the incident-side polarizer and the emitting-sidepolarizer are orthogonal to each other as illustrated in FIG. 2B. Here,as illustrated in FIG. 2B, in an “OFF” state where no voltage isapplied, light having passed through the incident-side polarizer passesthrough the emitting-side polarizer with the polarization axis rotatedby 90 degrees due to the effect of the liquid crystal layer. That is, ashutter in which no voltage is applied enters the light-transmittingstate.

On the other hand, as illustrated in FIG. 2B, in an “ON” state where avoltage is applied, since the polarization rotating effect due to liquidcrystal molecules of the liquid crystal layer disappears, light havingpassed through the incident-side polarizer is blocked by theemitting-side polarizer. That is, a shutter in which a voltage isapplied enters the light-blocking state.

Thus, for example, the infrared emitting unit emits infrared light in aperiod where a left-eye image is displayed on the monitor. Moreover, theinfrared receiving unit applies a voltage to the right-eye shutterwithout applying a voltage to the left-eye shutter in a period whereinfrared light is received. In this way, as illustrated in FIG. 2A,since the right-eye shutter enters the light-blocking state and theleft-eye shutter enters the light-transmitting state, a left-eye imageenters the left eye of the viewer. On the other hand, the infraredemitting unit stops emitting infrared light in a period where aright-eye image is displayed on the monitor. Moreover, the infraredreceiving unit applies a voltage to the left-eye shutter withoutapplying a voltage to the right-eye shutter in a period where noinfrared light is received. In this way, since the left-eye shutterenters the light-blocking state and the right-eye shutter enters thelight-transmitting state, a right-eye image enters the right eye of theviewer. As above, the stereoscopic display monitor illustrated in FIGS.2A and 2B displays images which can be perceived stereoscopically by theviewer by switching the images displayed on the monitor and the shutterstates in association. Another example of the stereoscopic displaymonitor which can display 2-parallax images stereoscopically is known,such as a monitor which employs polarized glasses method in addition tothe monitor which employs shutter method.

Furthermore, another example of stereoscopic display monitors that arecommercialized in recent years is known, such as a stereoscopic displaymonitor which enables multi-parallax images such as 9-parallax images tobe perceived stereoscopically with the naked eyes of a viewer by using abeam controller such as a lenticular lens. The stereoscopic displaymonitor enables stereoscopic display with a binocular parallax and alsoenables a video viewed with movement of the viewpoint of a viewer to beperceived stereoscopically with a varying movement parallax.

FIG. 3 is a diagram for describing an example of a stereoscopic displaymonitor that performs stereoscopic display using 9-parallax images. Inthe stereoscopic display monitor illustrated in FIG. 3, a beamcontroller is disposed on the front surface of a planar display surface200 such as a liquid crystal panel. For example, in the stereoscopicdisplay monitor illustrated in FIG. 3, a vertical lenticular sheet 201of which the optical aperture extends in the vertical direction isattached to the front surface of the display surface 200 as the beamcontroller. In the example illustrated in FIG. 3, although the verticallenticular sheet 201 is attached so that the convex portion thereof ison the front surface, the vertical lenticular sheet 201 may be attachedso that the convex portion faces the display surface 200.

As illustrated in FIG. 3, pixels 202 of which the aspect ratio is 3:1,and in which three sub pixels of the colors red (R), green (G), and blue(B) are arranged in the vertical direction are arranged on the displaysurface 200 in a matrix form. The stereoscopic display monitorillustrated in FIG. 3 converts 9-parallax images made up of nine imagesinto intermediate images arranged in a predetermined format (forexample, a grid form) and outputs the intermediate images to the displaysurface 200. That is, the stereoscopic display monitor illustrated inFIG. 3 outputs 9-parallax images by allocating the respective ninepixels located at the same positions as the 9-parallax images to ninecolumns of pixels 202. The nine columns of pixels 202 serve as unitpixels 203 that display nine images having different viewpoint positionssimultaneously.

The 9-parallax images output simultaneously to the display surface 200as the unit pixels 203 are radiated, for example, by an LED (lightemitting diode) backlight as parallel light and are further radiated bythe vertical lenticular sheet 201 in multiple directions. Since thelight of the respective pixels of the 9-parallax image is radiated inmultiple directions, the light entering the right and left eyes of aviewer changes in association with the position (the position of aviewpoint) of the viewer. That is, depending on the viewing angle of theviewer, the parallax image entering the right eye and the parallax imageentering the left eye have different parallax angles. In this way, theviewer can perceive a subject stereoscopically at each of the ninepositions illustrated in FIG. 3, for example. Moreover, the viewer canperceive the subject stereoscopically in a state of opposing in front ofthe subject at the position “5” illustrated in FIG. 3 and can perceivethe subject stereoscopically in a state of changing the orientation ofthe subject at the respective positions other than the position “5”illustrated in FIG. 3. The stereoscopic display monitor illustrated inFIG. 3 is an example only. The stereoscopic display monitor thatdisplays 9-parallax images may use a liquid crystal panel with ahorizontal stripe pattern of “RRR . . . , GGG . . . , BBB . . . ” asillustrated in FIG. 3, and may use a liquid crystal panel with avertical stripe pattern of “RGBRGB . . . .” Moreover, the stereoscopicdisplay monitor illustrated in FIG. 3 may use a vertical lens method inwhich the lenticular sheet is vertically arranged as illustrated in FIG.3, and may use an oblique lens method in which the lenticular sheet isobliquely arranged.

Hereinabove, a configuration example of the image processing system 1according to the first embodiment has been described briefly. Theapplication of the image processing system 1 is not limited to the casewhere the PACS is incorporated therein. For example, the imageprocessing system 1 is similarly applied to a case where an electronicchart system that manages an electronic chart to which medical imagesare attached is incorporated. In this case, the image archiving device120 is a database that archives electronic charts. Moreover, forexample, the image processing system 1 is similarly applied to a casewhere an HIS (hospital information system) or an RIS (radiologyinformation system) is incorporated. Moreover, the image processingsystem 1 is not limited to the configuration example described above.The functions possessed by the respective apparatuses and the allotmentthereof may be appropriately changed depending on the form ofapplication.

Next, a configuration example of the medical image diagnostic apparatus110 according to the first embodiment will be described with referenceto FIG. 4. FIG. 4 is a diagram for describing a configuration example ofthe medical image diagnostic apparatus 110 according to the firstembodiment.

As illustrated in FIG. 4, the medical image diagnostic apparatus 110according to the first embodiment includes a cradle unit 110 a and acalculator system unit 110 b. The cradle unit 110 a includes respectiveunits used for imaging, and for example, when the medical imagediagnostic apparatus 110 is an X-ray CT apparatus, the cradle unit 110 aincludes an X-ray tube, detectors, a rotating arm, a bed, and the like.On the other hand, the calculator system unit 110 b includes an inputunit 111, a display unit 112, a communication unit 113, a storage unit114, a control unit 115, and a rendering processing unit 116.

The input unit 111 is a mouse, a keyboard, a trackball, or the like, andreceives the input of various operations on the medical image diagnosticapparatus 110 from an operator. Specifically, the input unit 111according to the first embodiment receives the input of imaging plans,the input of imaging instructions, the input of conditions regardingrendering processes (hereinafter referred to as rendering conditions),and the like.

The display unit 112 is a liquid crystal panel or the like serving asthe stereoscopic display monitor and displays various types ofinformation. Specifically, the display unit 112 according to the firstembodiment displays a GUI (graphical user interface) for receivingvarious operations from an operator, a group of parallax images asdisplay images, and the like. The communication unit 113 is an NIC(network interface card) or the like and performs communication withother apparatuses.

The storage unit 114 is a hard disk, a semiconductor memory device, orthe like, and stores various types of information. Specifically, thestorage unit 114 according to the first embodiment stores imaging datacollected by imaging. Moreover, the storage unit 114 according to thefirst embodiment stores volume data generated from the imaging data,volume data under rendering processes, the group of parallax imagesgenerated by the rendering processes, and the like.

The control unit 115 is an electronic circuit such as a CPU (centralprocessing unit) or an MPU (micro processing unit), or an integratedcircuit such as an ASIC (application specific integrated circuit) or anFPGA (field programmable gate array), and performs control of the entiremedical image diagnostic apparatus 110.

For example, the control unit 115 according to the first embodimentcontrols the display of GUI and the display of the group of parallaximages on the display unit 112. Moreover, for example, the control unit115 controls the imaging performed by controlling the respective unitsincluded in the cradle unit 110 a and transmission/reception of volumedata or parallax images to/from the image archiving device 120,performed via the communication unit 113. Moreover, for example, thecontrol unit 115 controls the rendering processes by the renderingprocessing unit 116. Moreover, for example, the control unit 115controls the reading of various types of data from the storage unit 114and the storing of the data in the storage unit 114.

The rendering processing unit 116 performs various rendering processeson the volume data read from the storage unit 114 under the control ofthe control unit 115 to generate a group of parallax images.Specifically, first, the rendering processing unit 116 according to thefirst embodiment reads volume data from the storage unit 114 andperforms pre-processing on the volume data. Subsequently, the renderingprocessing unit 116 performs a volume rendering process on thepreprocessed volume data to generate a group of parallax images.Subsequently, the rendering processing unit 116 generates 2-dimensionalimages in which various types of information (scale, patient names,examination items, and the like) are drawn and superimposes thegenerated 2-dimensional images on each of the group of parallax imagesto thereby generate 2-dimensional output images. Moreover, the renderingprocessing unit 116 stores the generated group of parallax images andthe 2-dimensional output images in the storage unit 114. In the firstembodiment, the rendering process is the entire image processingperformed on the volume data, and the volume rendering process is aprocess of generating 2-dimensional images including 3-dimensionalinformation within the rendering process.

FIG. 5 is a diagram for describing a configuration example of therendering processing unit illustrated in FIG. 4. As illustrated in FIG.5, the rendering processing unit 116 includes a preprocessing unit 1161,a 3-dimensional image processing unit 1162, and a 2-dimensional imageprocessing unit 1163. The preprocessing unit 1161 performs preprocessingon volume data, the 3-dimensional image processing unit 1162 generates agroup of parallax images from the preprocessed volume data, and the2-dimensional image processing unit 1163 generates 2-dimensional outputimages in which various types of information are superimposed on thegroup of parallax images. Hereinafter, the respective units will bedescribed in order.

The preprocessing unit 1161 is a processing unit that performs varioustypes of preprocessing when performing rendering processes on volumedata. The preprocessing unit 1161 includes an image correctionprocessing unit 1161 a, a 3-dimensional object fusion unit 1161 e, and a3-dimensional object display region setting unit 1161 f.

The image correction processing unit 1161 a is a processing unit thatperforms an image correction process when processing two sets of volumedata as one set of volume data. As illustrated in FIG. 5, the imagecorrection processing unit 1161 a includes a distortion correctionprocessing unit 1161 b, a body motion correction processing unit 1161 c,and an inter-image registration processing unit 1161 d. For example, theimage correction processing unit 1161 a performs an image correctionprocess when processing the volume data of PET images and the volumedata of X-ray CT images generated by a PET-CT apparatus as one set ofvolume data. Alternatively, the image correction processing unit 1161 aperforms an image correction process when processing the volume data ofT1-enhanced images and the volume data of T2-enhanced images generatedby an MRI apparatus as one set of volume data.

Moreover, the distortion correction processing unit 1161 b corrects adata distortion in individual sets of volume data, resulting from thecollecting conditions when collecting data using the medical imagediagnostic apparatus 110. Moreover, the body motion correctionprocessing unit 1161 c corrects a movement resulting from a body motionof a subject when collecting data used for generating the individualsets of volume data. Moreover, the inter-image registration processingunit 1161 d performs registration between two sets of volume data whichhave been corrected by the distortion correction processing unit 1161 band the body motion correction processing unit 1161 c, using across-correlation method or the like, for example.

The 3-dimensional object fusion unit 1161 e fuses multiple sets ofvolume data which have been registered by the inter-image registrationprocessing unit 1161 d. The processes of the image correction processingunit 1161 a and the 3-dimensional object fusion unit 1161 e are notperformed when the rendering process is performed with respect to asingle set of volume data.

The 3-dimensional object display region setting unit 1161 f is aprocessing unit that sets a display region corresponding to a displaytarget organ designated by an operator. The 3-dimensional object displayregion setting unit 1161 f includes a segmentation processing unit 1161g. The segmentation processing unit 1161 g is a processing unit thatextracts an organ such as the heart, the lung, or blood vessels,designated by an operator using a region growing method based on thepixel values (voxel values) of volume data, for example.

The segmentation processing unit 1161 g does not perform a segmentationprocess when a display target organ is not designated by the operator.Moreover, when multiple display target organs are designated by theoperator, the segmentation processing unit 1161 g extracts thecorresponding multiple organs. Moreover, the processing of thesegmentation processing unit 1161 g may be executed over and over inresponse to a fine adjustment request from an operator who referred to arendering image.

The 3-dimensional image processing unit 1162 performs a volume renderingprocess on the volume data preprocessed by the preprocessing unit 1161.As processing units that perform the volume rendering process, the3-dimensional image processing unit 1162 includes a projection methodsetting unit 1162 a, a 3-dimensional geometry conversion processing unit1162 b, a 3-dimensional object appearance processing unit 1162 f, and a3-dimensional virtual space rendering unit 1162 k.

The projection method setting unit 1162 a determines a projection methodfor generating a group of parallax images. For example, the projectionmethod setting unit 1162 a determines whether the volume renderingprocess is to be executed by a parallel projection method or aperspective projection method.

The 3-dimensional geometry conversion processing unit 1162 b is aprocessing unit that determines information for geometrically convertingthe volume data to be subjected to the volume rendering process into a3-dimensional space. The 3-dimensional geometry conversion processingunit 1162 b includes a parallel shift processing unit 1162 c, a rotationprocessing unit 1162 d, and a zoom processing unit 1162 e. The parallelshift processing unit 1162 c is a processing unit that determines theamount of parallel shift of volume data when the viewpoint position isshifted in parallel during the volume rendering process. The rotationprocessing unit 1162 d is a processing unit that determines the amountof rotational movement of volume data when the viewpoint position isrotated during the volume rendering process. Moreover, the zoomprocessing unit 1162 e is a processing unit that determines the zoomratio of volume data when it is requested to zoom in or out the group ofparallax images.

The 3-dimensional object appearance processing unit 1162 f includes a3-dimensional object color processing unit 1162 g, a 3-dimensionalobject opacity processing unit 1162 h, a 3-dimensional object textureprocessing unit 1162 i, and a 3-dimensional virtual space light sourceprocessing unit 1162 j. The 3-dimensional object appearance processingunit 1162 f performs a process of determining the display state of thegroup of parallax images displayed in accordance with the request of theoperator, for example, using these processing units.

The 3-dimensional object color processing unit 1162 g is a processingunit that determines the colors to be filled in the respective segmentedregions of the volume data. The 3-dimensional object opacity processingunit 1162 h is a processing unit that determines the opacity ofrespective voxels constituting each of the segmented regions of thevolume data. Regions of the volume data behind a region having opacityof “100%” are not drawn in the group of parallax images. Moreover,regions of the volume data having opacity of “0%” are not drawn in thegroup of parallax images.

The 3-dimensional object texture processing unit 1162 i is a processingunit that determines the material of respective segmented regions of thevolume data to thereby adjust the texture when the regions are drawn.The 3-dimensional virtual space light source processing unit 1162 j is aprocessing unit that determines the position of a virtual light sourcedisposed in a 3-dimensional virtual space and the type of the virtuallight source when performing the volume rendering process on the volumedata. Examples of the type of the virtual light source include a lightsource that emits parallel light beams from the infinity and a lightsource that emits radiating light beams from a viewpoint.

The 3-dimensional virtual space rendering unit 1162 k performs thevolume rendering process to the volume data to thereby generate a groupof parallax images. Moreover, when performing the volume renderingprocess, the 3-dimensional virtual space rendering unit 1162 k usesvarious types of information determined by the projection method settingunit 1162 a, the 3-dimensional geometry conversion processing unit 1162b, and the 3-dimensional object appearance processing unit 1162 f asnecessary.

Here, the volume rendering process by the 3-dimensional virtual spacerendering unit 1162 k is performed in accordance with the renderingcondition. For example, the rendering condition is “parallel projectionmethod” or “perspective projection method.” Moreover, for example, therendering condition is “reference viewpoint position and parallaxangle.” Moreover, for example, the rendering condition is “parallelshift of viewpoint position,” “rotational movement of viewpointposition,” “zoom-in of group of parallax images,” and “zoom-out of groupof parallax images.” Moreover, for example, the rendering condition is“filling color,” “transparency,” “texture,” “position of virtual lightsource,” and “type of virtual light source.” These rendering conditionsmay be received from the operator via the input unit 111 and may be setas initial settings. In any cases, the 3-dimensional virtual spacerendering unit 1162 k receives the rendering conditions from the controlunit 115 and performs the volume rendering process on the volume data inaccordance with the rendering conditions. Moreover, in this case, sincethe projection method setting unit 1162 a, the 3-dimensional geometryconversion processing unit 1162 b, and the 3-dimensional objectappearance processing unit 1162 f determine various types of necessaryinformation in accordance with the rendering conditions, the3-dimensional virtual space rendering unit 1162 k generates a group ofparallax images using these various types of determined information.

FIG. 6 is a diagram for describing an example of the volume renderingprocess according to the first embodiment. For example, as illustratedin “9-parallax image generation method (1)” of FIG. 6, it is assumedthat the 3-dimensional virtual space rendering unit 1162 k receives aparallel projection method as the rendering condition and receives areference viewpoint position of (5) and a parallax angle of “1 degree.”In this case, the 3-dimensional virtual space rendering unit 1162 kshifts the viewpoint positions in parallel to the positions of (1) to(9) so that the parallax angle is “1 degree” to thereby generate nineparallax images by the parallel projection method so that the differencebetween parallax angles (the angle between the eye directions) is 1degree. When performing the parallel projection method, the3-dimensional virtual space rendering unit 1162 k sets a light sourcethat emits parallel light beams from the infinity along the eyedirection.

Alternatively, as illustrated in “9-parallax image generation method(2)” of FIG. 6, it is assumed that the 3-dimensional virtual spacerendering unit 1162 k receives a perspective projection method as therendering condition and receives a reference viewpoint position of (5)and a parallax angle of “1 degree.” In this case, the 3-dimensionalvirtual space rendering unit 1162 k rotates the viewpoint position aboutthe center (weighted center) of the volume data to the positions of (1)to (9) so that the parallax angle is “1 degree” to thereby generate nineparallax images by the perspective projection method so that thedifference between parallax angles is 1 degree. When performing theperspective projection method, the 3-dimensional virtual space renderingunit 1162 k sets a point light source or a field light source that emitslight 3-dimensionally in a radial form about the eye direction atrespective viewpoints. Moreover, when performing the perspectiveprojection method, the viewpoints (1) to (9) may be shifted in paralleldepending on the rendering condition.

The 3-dimensional virtual space rendering unit 1162 k may perform thevolume rendering process using both the parallel projection method andthe perspective projection method by setting a light source that emitslight 2-dimensionally in a radial form about the eye direction withrespect to the vertical direction of volume rendering images to bedisplayed and emits parallel light beams from the infinity along the eyedirection with respect to the horizontal direction of the volumerendering images to be displayed.

The nine parallax images generated in this way are the group of parallaximages. In the first embodiment, the nine parallax images are convertedinto intermediate images arranged in a predetermined format (forexample, a grid form) by the control unit 115, for example, and areoutput to the display unit 112 as the stereoscopic display monitor.Then, the operator of the workstation 130 can perform operations forgenerating the group of parallax images while checking the stereoscopicmedical images displayed on the stereoscopic display monitor.

In the example of FIG. 6, although a case where a projection method, areference viewpoint position, and a parallax angle are received as therendering conditions has been described, even when other conditions arereceived as the rendering conditions, the 3-dimensional virtual spacerendering unit 1162 k generates a group of parallax images whilereflecting the respective rendering conditions in the same manner.

Moreover, the 3-dimensional virtual space rendering unit 1162 k also hasa function of performing an MPR (multi-planar reconstruction) method aswell as the volume rendering method to reconstruct MPR images fromvolume data. The 3-dimensional virtual space rendering unit 1162 k has afunction of performing “curved MPR” and a function of performing“intensity projection.”

Subsequently, the group of parallax images generated from the volumedata by the 3-dimensional image processing unit 1162 is set as anunderlay. Moreover, an overlay in which various types of information(scale, patient names, examination items, and the like) are drawn issuperimposed on the underlay, and the resulting images are set as2-dimensional output images. The 2-dimensional image processing unit1163 is a processing unit that generates 2-dimensional output images byperforming image processing on the overlay and the underlay. Asillustrated in FIG. 5, the 2-dimensional image processing unit 1163includes a 2-dimensional object rendering unit 1163 a, a 2-dimensionalgeometry conversion processing unit 1163 b, and a brightness adjustmentunit 1163 c. For example, the 2-dimensional image processing unit 1163generates nine 2-dimensional output images by superimposing one overlayto each of nine parallax images (underlays) in order to decrease theload necessary for the process of generating 2-dimensional outputimages.

The 2-dimensional object rendering unit 1163 a is a processing unit thatrenders various types of information to be drawn on the overlay. The2-dimensional geometry conversion processing unit 1163 b is a processingunit that performs the process of shifting in parallel or rotationallymoving the positions of various types of information drawn on theoverlay and the process of zooming in or out various types ofinformation drawn on the overlay.

Moreover, the brightness adjustment unit 1163 c is a processing unitthat performs a brightness conversion process, and for example, is aprocessing unit that adjusts the brightness of the overlay and theunderlay in accordance with image processing parameters such asgradation of a stereoscopic display monitor at the output destination, aWW (window width), or a WL (window level).

The 2-dimensional output images generated in this way are stored in thestorage unit 114 by the control unit 115, for example, and are thentransmitted to the image archiving device 120 via the communication unit113. For example, when the workstation 130 or the terminal device 140acquires the 2-dimensional output images from the image archiving device120 and displays the same on the stereoscopic display monitor afterconverting into intermediate images arranged in a predetermined format(for example, a grid form), doctors or examiners who are viewers canview stereoscopic medical images in which various types of information(scale, patient names, examination items, and the like) are drawn.

Meanwhile, as described above, the medical image diagnostic apparatus110 according to the first embodiment also displays icons displayed onthe operation screen so as to be perceived stereoscopically. Moreover,the icons displayed stereoscopically in the first embodiment are icons(hereinafter, thumbnail icons) for allowing operators to select medicalimage data.

From the past, the medical image diagnostic apparatus 110 has displayedthumbnail icons on the operation screen in order to allow operators toselect medical image data. For example, thumbnail icons which aredesigns for reduced medical images or thumbnail icons which are designsfor artificial medical images have been displayed. However, the medicalimage data stored in the medical image diagnostic apparatus 110 includesvarious types of data including volume data, 2-dimensional images afterrendering processes, a group of parallax images, and a group of2-dimensional output images in which various types of information aresuperimposed on each of the group of parallax images. Thus, when onlythe thumbnail icons of reduced medical images or artificial medicalimages are just displayed on the operation screen, it was difficult forthe operators to understand the type of medical image data.

In this regard, the medical image diagnostic apparatus 110 according tothe first embodiment determines whether thumbnail icons will bedisplayed as images (hereinafter referred to as stereoscopic images)which can be perceived stereoscopically or as the other images(hereinafter referred to as planar images) depending on the content ofthe medical image data selected by the operator. Moreover, whenthumbnail icons are displayed as stereoscopic images, the medical imagediagnostic apparatus 110 according to the first embodiment displaysthumbnail icons so that the stereoscopic effect of the thumbnail iconreflects the quantity of medical image data (for example, an imagingrange in the body axis direction included in the medical image data, andthe number of medical image data included in volume data when themedical image data is volume data). As a result, the operation screen isdisplayed appropriately, and the operator can understand the type ofimage and the quantity of medical image data just by watching thumbnailicons. This will be described in detail below.

FIGS. 7 and 8 are diagrams for describing a display example according tothe first embodiment. In FIGS. 7 and 8, a region (hereinafter referredto as a thumbnail region) for displaying thumbnail icons, extracted fromthe operation screen is illustrated for the sake of convenience. Asillustrated in FIG. 7, although the medical image diagnostic apparatus110 displays multiple thumbnail icons to be aligned, thumbnail icons a1,a2, and a4 are displayed as stereoscopic images, and a thumbnail icon a3is displayed as a planar image. Moreover, as illustrated in FIG. 8, themedical image diagnostic apparatus 110 displays the thumbnail icons sothat the thumbnail icon a4 has a greater stereoscopic effect than a1,and the thumbnail icon a2 has a greater stereoscopic effect than a4.

Although the stereoscopic effect in the first embodiment means the senseof frontward protrusion from a reference surface (the background screenin FIGS. 7 and 8) of the operation screen, the embodiments are notlimited to this, but the stereoscopic effect may be the sense of depthin the depth direction from the reference surface or a combination ofthe sense of protrusion and the sense of depth.

FIG. 9 is a diagram for describing a configuration example of thestorage unit 114 and the control unit 115 according to the firstembodiment. As illustrated in FIG. 9, the storage unit 114 of themedical image diagnostic apparatus 110 according to the first embodimentincludes an imaging data storage unit 114 a, a medical image datastorage unit 114 b, and an operation screen information storage unit 114c.

The imaging data storage unit 114 a stores imaging data in accordancewith the operation of an imaging unit 115 a described later. Moreover,the imaging data stored by the imaging data storage unit 114 a is usedfor the processing by a reconstructing unit 115 b described later.

The medical image data storage unit 114 b stores volume data,2-dimensional images after rendering processes, a group of parallaximages, a group of 2-dimensional output images in which various types ofinformation are superimposed on each of the group of parallax images,and the like in accordance with the operation of the reconstructing unit115 b described later. The medical image data stored by the medicalimage data storage unit 114 b is used for the processing by a medicalimage data information acquiring unit 115 d and a display control unit115 f described later.

The operation screen information storage unit 114 c stores basicinformation for displaying the operation screen, which is stored inadvance, for example, when the medical image diagnostic apparatus 110 isset up, and stores operation screen display information in whichthumbnail icons or the like are added to basic information in accordancewith the operation of an operation screen generating unit 115 edescribed later. Moreover, the basic information stored by the operationscreen information storage unit 114 c is used for the processing by theoperation screen generating unit 115 e, and the operation screen displayinformation stored by the operation screen information storage unit 114c is used for the processing by the display control unit 115 f.

Next, as illustrated in FIG. 9, the control unit 115 of the medicalimage diagnostic apparatus 110 according to the first embodimentincludes the imaging unit 115 a, the reconstructing unit 115 b, anoperation receiving unit 115 c, the medical image data informationacquiring unit 115 d, the operation screen generating unit 115 e, andthe display control unit 115 f.

The imaging unit 115 a performs imaging by controlling the respectiveunits of the cradle unit 110 a in accordance with predetermined imagingconditions. Moreover, the imaging unit 115 a stores the imaging datacollected by imaging in the imaging data storage unit 114 a. Forexample, when the medical image diagnostic apparatus 110 is an X-ray CTapparatus, the imaging unit 115 a collects projection data bycontrolling an X-ray tube, detectors, a rotating arm, and the like inaccordance with predetermined imaging conditions and stores thecollected projection data in the imaging data storage unit 114 a.

The reconstructing unit 115 b reads imaging data from the imaging datastorage unit 114 a and performs a reconstructing process on the readimaging data to thereby generate volume data. Moreover, thereconstructing unit 115 b performs a rendering process on the generatedvolume data in cooperation with the rendering processing unit 116 tothereby generate 2-dimensional images after the rendering process, agroup of parallax images, a group of 2-dimensional output images inwhich various types of information are superimposed on each of the groupof parallax images, and the like. Moreover, the reconstructing unit 115b stores the generated volume data, 2-dimensional images after therendering process, group of parallax images, group of 2-dimensionaloutput images, and the like in the medical image data storage unit 114b.

The operation receiving unit 115 c receives operations via the inputunit 111. For example, the operation receiving unit 115 c receives anoperation for an operation screen display instruction. In this case, theoperation receiving unit 115 c notifies the medical image datainformation acquiring unit 115 d of the note of the reception of theoperation screen display instruction together with a patient ID and anexamination ID input at an imaging planning step, for example. Moreover,for example, the operation receiving unit 115 c receives an operation ofselecting a thumbnail icon displayed on the operation screen. In thiscase, the operation receiving unit 115 c notifies the display controlunit 115 f of the selected thumbnail icon.

The medical image data information acquiring unit 115 d acquires medicalimage data information necessary for generating a thumbnail regiondisplayed on the operation screen. Specifically, upon receiving thenotification of the reception of the operation for the operation screendisplay instruction from the operation receiving unit 115 c, the medicalimage data information acquiring unit 115 d specifies the correspondingmedical image data by referring to the medical image data storage unit114 b using a patient ID and an examination ID, for example.Subsequently, the medical image data information acquiring unit 115 dacquires the type of medical image data, an imaging range in the bodyaxis direction included in the medical image data, and the number ofmedical image data included in volume data when the medical image datais volume data, for example, as the medical image data information ofthe specified medical image data. Moreover, the medical image datainformation acquiring unit 115 d notifies the operation screengenerating unit 115 e of the acquired medical image data information.The medical image data information is one which is stored in the medicalimage data storage unit 114 b, and in which the information input at animaging planning step, for example, is stored together with imagingdata, volume data generated later, and the like.

The operation screen generating unit 115 e generates operation screendisplay information. Specifically, upon receiving the notification ofmedical image data information from the medical image data informationacquiring unit 115 d, the operation screen generating unit 115 eacquires basic information for displaying the operation screen byreferring to the operation screen information storage unit 114 c andgenerates a thumbnail region based on the medical image data informationto thereby generate operation screen display information in whichthumbnail icons or the like are added to the basic information.Moreover, the operation screen generating unit 115 e stores thegenerated operation screen display information in the operation screeninformation storage unit 114 c and notifies the display control unit 115f of the note indicating that the operation screen display informationhas been generated.

Generation of the thumbnail region by the operation screen generatingunit 115 e will be described in detail. FIGS. 10 and 11 are diagrams fordescribing generation of the thumbnail region according to the firstembodiment. For example, the operation screen generating unit 115 estores a table (see FIGS. 10 and 11) for generating thumbnail icons, anda group of parallax images for thumbnail icons and rectangular planarimages for thumbnail icons generated in advance by performingcomputation processing so that a cuboid can be perceivedstereoscopically. Moreover, the operation screen generating unit 115 estores multiple patterns of groups of parallax images of which theheights of cuboids (frontward heights from the reference surface of theoperation screen) are different so that the stereoscopic effect ofthumbnail icons reflects the quantity of medical image data. Forexample, the multiple patterns of groups of parallax images are multiplepatterns of groups of parallax images having different parallax angles,and generally, the stereoscopic effect increases as the parallax angleincreases.

As illustrated in FIG. 10, the operation screen generating unit 115 estores the correspondence between an imaging range and a thumbnail iconimage as the table for generating thumbnail icons. For example, theoperation screen generating unit 115 e stores a thumbnail icon image“head.jpeg” in correlation with an imaging range “head.” The thumbnailicon image is one image or a representative image included in medicalimage data, for example. For example, when medical image data of 500axial planes are included in the medical image data, the thumbnail iconimage is medical image data of the first axial plane or medical imagedata of the 250th axial plane corresponding to the center of the imagingrange. Alternatively, the thumbnail icon image may be an MPR imagegenerated from the medical image data. Alternatively, the thumbnail iconimage may be medical image data collected at the time of scanogramscanning.

Moreover, as illustrated in FIG. 11, the operation screen generatingunit 115 e stores the correspondence between the number of medical imagedata and a stereoscopic effect as the table for generating thumbnailicons. For example, the operation screen generating unit 115 e stores astereoscopic effect “2” in correlation with the number “upto 200 (from101 to 200).” The example illustrated in FIG. 11 represents that thestereoscopic effect increases as the number assigned to the stereoscopiceffect increases. Moreover, in the first embodiment, although thestereoscopic effect is reflected based on the number of medical imagedata, the embodiments are not limited to this, but the stereoscopiceffect may be reflected based on the imaging range. For example, thestereoscopic effect of the imaging range “whole body” may be greaterthan the stereoscopic effect of the imaging range “head.”

Meanwhile, as described above, the operation screen generating unit 115e according to the first embodiment receives the type of medical imagedata, the imaging range in the body axis direction included in themedical image data, and the number of medical image data included involume data when the medical image data is volume data, as the medicalimage data information.

For example, the operation screen generating unit 115 e receives thetype of medical image data “volume data,” the imaging range in body axisdirection “head,” and the number of medical image data “200” as firstmedical image data information. Moreover, the operation screengenerating unit 115 e receives the type of medical image data “MPRimage,” the imaging range in body axis direction “head,” and the numberof medical image data “1” as second medical image data information.Moreover, the operation screen generating unit 115 e receives the typeof medical image data “volume data,” the imaging range in body axisdirection “abdomen,” and the number of medical image data “500” as thirdmedical image data information. Moreover, the operation screengenerating unit 115 e receives the type of medical image data “MPRimage,” the imaging range in body axis direction “abdomen,” and thenumber of medical image data “1” as fourth medical image datainformation.

Then, the operation screen generating unit 115 e determines that athumbnail icon image “head.jpeg” correlated with the imaging range“head” is used as an image to be attached to the surface of thethumbnail icon with respect to the first medical image data informationby referring to the table illustrated in FIG. 10. Moreover, theoperation screen generating unit 115 e determines that “2” is selectedas the stereoscopic effect of the thumbnail icon by referring to thetable illustrated in FIG. 11. Moreover, the operation screen generatingunit 115 e acquires a group of parallax images in which a cuboid havingthe height corresponding to the stereoscopic effect “2” can be perceivedstereoscopically and attaches the thumbnail icon image “head.jpeg” toeach of the parallax images included in the group of parallax images.

Moreover, the operation screen generating unit 115 e determines that athumbnail icon image “head.jpeg” correlated with the imaging range“head” is used as an image to be attached to the surface of thethumbnail icon with respect to the second medical image data informationby referring to the table illustrated in FIG. 10. Moreover, theoperation screen generating unit 115 e determines that “0” is selectedas the stereoscopic effect of the thumbnail icon by referring to thetable illustrated in FIG. 11. Moreover, the operation screen generatingunit 115 e acquires a rectangular planar image corresponding to thestereoscopic effect “0” and attaches the thumbnail icon image“head.jpeg” to the planar image.

Moreover, the operation screen generating unit 115 e determines that athumbnail icon image “abdomen.jpeg” correlated with the imaging range“abdomen” is used as an image to be attached to the surface of thethumbnail icon with respect to the third medical image data informationby referring to the table illustrated in FIG. 10. Moreover, theoperation screen generating unit 115 e determines that “5” is selectedas the stereoscopic effect of the thumbnail icon by referring to thetable illustrated in FIG. 11. Moreover, the operation screen generatingunit 115 e acquires a group of parallax images in which a cuboid havingthe height corresponding to the stereoscopic effect “5” can be perceivedstereoscopically and attaches the thumbnail icon image “abdomen.jpeg” toeach of the parallax images included in the group of parallax images.

Moreover, the operation screen generating unit 115 e determines that athumbnail icon image “abdomen.jpeg” correlated with the imaging range“abdomen” is used as an image to be attached to the surface of thethumbnail icon with respect to the fourth medical image data informationby referring to the table illustrated in FIG. 10. Moreover, theoperation screen generating unit 115 e determines that “0” is selectedas the stereoscopic effect of the thumbnail icon by referring to thetable illustrated in FIG. 11. Moreover, the operation screen generatingunit 115 e acquires a rectangular planar image corresponding to thestereoscopic effect “0” and attaches the thumbnail icon image“abdomen.jpeg” to the planar image.

Moreover, the operation screen generating unit 115 e determines a regionwhere respective thumbnail icons are arranged so that the first tofourth thumbnail icons are appropriately arranged in the thumbnailregion on the optical screen. Here, the stereoscopic display monitorincluded in the medical image diagnostic apparatus 110 can displaystereoscopic images by displaying a group of parallax images having apredetermined parallax number and can also display planar images byreplacing a plurality of identical images with a group of parallaximages and displaying the same. For example, as illustrated in FIG. 3,the stereoscopic display monitor according to the first embodiment candisplay stereoscopic images by outputting nine pixels of a 9-parallaximage located at the same position by allocating the same to ninecolumns of pixels 202 and can also display planar images by outputtingone pixel of the nine pixels by allocating the same to all of the ninecolumns of pixels 202.

Thus, for example, the operation screen generating unit 115 e accordingto the first embodiment generates operation screen information so thatthe same pixels are allocated to all of the nine columns of pixels 202with respect to a region of the operation screen other than the regionwhere stereoscopic thumbnail icons or stereoscopic images are displayed.On the other hand, the operation screen generating unit 115 e generatesoperation screen information so that nine pixels located at the sameposition in each of the parallax images included in a group of parallaximages are allocated to the respective nine columns of pixels 202 withrespect to the region of the operation screen where the stereoscopicthumbnail icons or stereoscopic images are displayed.

The above-described method of generating operation screen information isan example only. For example, an operation screen of a different layermay be generated for respective display regions, thumbnail regions, andthumbnail icons, the operation screens of respective layers may besubjected to necessary mask processing, and the respective processedlayers may be disposed in a superimposed manner. In this case, the maskprocessing is performed such that the opacity of a region whereinformation is disposed is set to 100%, and the opacity of a regionwhere no information is disposed is set to 0%.

The display control unit 115 f displays an operation screen on thedisplay unit 112. Specifically, upon receiving the notification ofgeneration of the operation screen display information from theoperation screen generating unit 115 e, the display control unit 115 facquires operation screen display information by referring to theoperation screen information storage unit 114 c. Moreover, the displaycontrol unit 115 f displays an operation screen on the display unit 112using the acquired operation screen display information, and ifnecessary, the medical image data stored in the medical image datastorage unit 114 b.

FIG. 12 is a diagram for describing an operation screen according to thefirst embodiment. The operation screen illustrated in FIG. 12 includes adisplay region A, a display region B, a display region C, and athumbnail region. As illustrated in FIG. 12, the display region A is aregion for displaying a registration image which is set in advance to bedisplayed on an initial screen. Moreover, for example, the displayregion B is a region for displaying a medical image corresponding tomedical image data selected in a thumbnail region. For example, thedisplay region C is a region for displaying the other sets of operationinformation. In the example illustrated in FIG. 12, a registration imagecorresponding to the head is displayed in the display region A, andnothing is displayed in the display region B.

Moreover, as illustrated in FIG. 12, the thumbnail region is a regionfor displaying thumbnail images. In the example illustrated in FIG. 12,thumbnail icons of medical image data collected by scanogram scanning,thumbnail icons of registration images, and thumbnail icons of medicalimage data collected by main imaging are displayed in the thumbnailregion. The first to fourth thumbnail icons described above aredisplayed as “thumbnail icons of medical image data collected by mainimaging” in the thumbnail region.

Moreover, the display control unit 115 f updates the operation screen.Specifically, upon receiving the notification of the selected thumbnailicon from the operation receiving unit 115 c, the display control unit115 f updates the operation screen in accordance with the selectedthumbnail icon. For example, when the selected thumbnail icon meansselecting certain volume data, the display control unit 115 f displays ascreen for receiving the input of rendering conditions for performing arendering process using the volume data in another window different fromthe operation screen, for example. Moreover, for example, when theselected thumbnail icon means selecting a certain MPR image, the displaycontrol unit 115 f acquires the MPR image from the medical image datastorage unit 114 b and displays the same in the display region B.

Subsequently, FIGS. 13 and 14 are flowcharts illustrating the flow of adisplay control process according to the first embodiment. Asillustrated in FIG. 13, when the operation receiving unit 115 c receivesan operation for an operation screen display instruction (Yes in stepS101), the medical image data information acquiring unit 115 d acquiresmedical image data information from the medical image data storage unit114 b (step S102). Subsequently, the operation screen generating unit115 e generates a thumbnail region (step S103) and generates operationscreen display information (step S104). Moreover, the display controlunit 115 f displays an operation screen on the display unit 112 (stepS105).

Moreover, as illustrated in FIG. 14, when the operation receiving unit115 c receives an operation of selecting a thumbnail icon (Yes in stepS201), and the selected thumbnail icon means selecting volume data (Yesin step S202), the display control unit 115 f displays a screen forreceiving the input of rendering conditions in another window differentfrom the operation screen, for example (step S203). On the other hand,when the selected thumbnail icon means selecting data other than volumedata (No in step S202), the display control unit 115 f acquires thecorresponding medical image data from the medical image data storageunit 114 b and displays the same in the display region B, for example(step S204).

Advantageous Effects of First Embodiment

As described above, according to the first embodiment, since whetherthumbnail icons will be displayed as stereoscopic images or planarimages is determined in accordance with the content of medical imagedata selected by the operator, the operation screen is displayedappropriately. Thus, the operator can understand the type of an imagejust by watching thumbnail icons. Moreover, according to the firstembodiment, since medical image data is displayed so that thestereoscopic effect of thumbnail icons reflects the quantity of medicalimage data, the operation screen is displayed appropriately. Thus, theoperator can understand the quantity of medical image data just bywatching thumbnail icons.

Modification Example of First Embodiment

Next, a modification example of the first embodiment will be described.In the first embodiment, a method in which a thumbnail icon imagecorresponding to the imaging range is displayed to be attached to thesurface of a cuboid as a thumbnail icon has been described. However, theembodiments are not limited to this.

In this modification example, a method of displaying a stereoscopicimage generated by performing a rendering process on volume data as athumbnail icon will be described.

FIGS. 15 and 16 are diagrams for describing an operation screenaccording to a modification example of the first embodiment. In FIG. 16,a part of a thumbnail region extracted from an operation screen isillustrated for the sake of convenience.

As illustrated in FIG. 15, the medical image diagnostic apparatus 110displays thumbnail icons b1 and b3 as stereoscopic images and displaysthumbnail icons b2 and b4 as planar images. Here, the thumbnail icons b1and b3 illustrated in FIG. 15 are stereoscopic images generated byperforming a rendering process on volume data. As illustrated in FIG.16, for example, the thumbnail icon b1 is a stereoscopic image of brainblood vessels generated from the volume data of the head. Moreover, forexample, the thumbnail icon b3 is a stereoscopic image of bonesgenerated from the volume data of the abdomen.

Generation of thumbnail regions by the operation screen generating unit115 e according to this modification example will be described indetail. For example, the operation screen generating unit 115 e receivesthe type of medical image data “volume data” and the imaging range inbody axis direction “head” as first medical image data information.Moreover, the operation screen generating unit 115 e receives the typeof medical image data “MPR image” and the imaging range in body axisdirection “head” as second medical image data information. Moreover, theoperation screen generating unit 115 e receives the type of medicalimage data “volume data” and the imaging range in body axis direction“abdomen” as third medical image data information. Moreover, theoperation screen generating unit 115 e receives the type of medicalimage data “MPR image” and the imaging range in body axis direction“abdomen” as fourth medical image data information.

Then, since the type of medical image data for the first medical imagedata information is “volume data,” the operation screen generating unit115 e determines to generate stereoscopic images and acquires thecorresponding volume data by referring to the medical image data storageunit 114 b. Moreover, in the first embodiment, it is set in advance thatwhen generating stereoscopic images for thumbnail icons from the volumedata of the imaging range “head,” the operation screen generating unit115 e segments brain blood vessels and generates an image of the brainblood vessels. Moreover, the rendering conditions for generatingstereoscopic images are set in advance. Thus, the operation screengenerating unit 115 e performs the rendering process by the renderingprocessing unit 116 with respect to the acquired volume data to therebygenerate stereoscopic images, that is, a group of parallax images of thebrain blood vessels. The operation screen generating unit 115 egenerates a group of parallax images so as to have the size of a regionused as thumbnail icons.

Similarly, since the type of medical image data for the third medicalimage data information is “volume data,” the operation screen generatingunit 115 e determines to generate stereoscopic images and acquires thecorresponding volume data by referring to the medical image data storageunit 114 b. Moreover, in the first embodiment, it is set in advance thatwhen generating stereoscopic images for thumbnail icons from the volumedata of the imaging range “abdomen,” the operation screen generatingunit 115 e segments bones and generates an image of the bones. Moreover,the rendering conditions for generating stereoscopic images are set inadvance. Thus, the operation screen generating unit 115 e performs therendering process by the rendering processing unit 116 with respect tothe acquired volume data to thereby generate stereoscopic images, thatis, a group of parallax images of the bones. The operation screengenerating unit 115 e generates a group of parallax images so as to havethe size of a region used as thumbnail icons.

As for the second and fourth medical image data information, theoperation screen generating unit 115 e may generate thumbnail icons ofplanar images by the same method as the first embodiment. Moreover,similarly to the first embodiment described above, the operation screengenerating unit 115 e determines the region for arranging the respectivethumbnail icons so that the first to fourth thumbnail icons areappropriately arranged in the thumbnail region on the operation screenand generates operation screen information so that the same pixels areallocated to all of the nine columns of pixels 202 with respect toregion of the operation screen other than the region where thestereoscopic thumbnail icons and stereoscopic images are displayed. Onthe other hand, the operation screen generating unit 115 e generatesoperation screen information so that nine pixels located at the sameposition in each of the parallax images included in a group of parallaximages are allocated to the respective nine columns of pixels 202 withrespect to the region of the operation screen where the stereoscopicthumbnail icons or stereoscopic images are displayed.

Other Modification Examples

Moreover, the embodiments are not limited to the above modificationexample. For example, in FIGS. 15 and 16, although thumbnail icons arearrange horizontally, the embodiments are not limited to this, but thethumbnail icons may be arranged vertically. For example, a thumbnailicon of “head,” a thumbnail icon of “chest,” a thumbnail icon of“abdomen,” and a thumbnail icon of “leg” may be arranged vertically sothat the imaging range can be understood immediately.

Moreover, a stereoscopic image which is a body mark prepared in advancemay be displayed as a thumbnail icon, for example. In this case, theoperation screen generating unit 115 e stores the thumbnail icon b1 ofFIG. 15 in advance as the thumbnail icon for the volume data of “head,”for example, and stores the thumbnail icon b3 of FIG. 15 in advance asthe thumbnail icon of the volume data of “abdomen,” for example.Moreover, the operation screen generating unit 115 e may select thecorresponding thumbnail icon based on medical image data information anddisplay the thumbnail icon.

Second Embodiment

Next, a second embodiment will be described. Although thumbnail iconsare displayed so as to be perceived stereoscopically in the firstembodiment, a message notified to an operator is displayed so as to beperceived stereoscopically in the second embodiment.

From the past, the medical image diagnostic apparatus 110 has displayeda message notified to the operator on the operation screen usingcharacters, a pop-up window, or the like. However, if the message isdisplayed as characters or a pop-up window, the operator may miss thepresence thereof.

In this regard, the medical image diagnostic apparatus 110 according tothe second embodiment determines whether the message will be displayedas a stereoscopic image or as a planar image depending on the content ofthe message. That is, when the message notified to the operator is setas a message of high importance, the medical image diagnostic apparatus110 displays the message as a stereoscopic image. As a result, theoperation screen is displayed appropriately so that medical informationthat should attract the operator's attention appears prominent, and theoperator cannot miss the message of high importance. This will bedescribed below.

FIG. 17 is a diagram for describing a display example according to thesecond embodiment. As illustrated in FIG. 17, the medical imagediagnostic apparatus 110 displays a warning message as a stereoscopicimage, for example. Although the stereoscopic effect of the stereoscopicimage in the second embodiment means the sense of frontward protrusionfrom a reference surface (the background screen in FIG. 17) of theoperation screen, the embodiments are not limited to this, but thestereoscopic effect may be the sense of depth in the depth directionfrom the reference surface or a combination of the sense of protrusionand the sense of depth.

For example, the medical image diagnostic apparatus 110 stores varioustypes of messages and the importance thereof in advance in correlation.Moreover, upon receiving a message display instruction from the imagingunit 115 a or the reconstructing unit 115 b, for example, the displaycontrol unit 115 f controls so that the message is displayed as astereoscopic image by referring to the importance stored in correlationwith the message when the importance thereof exceeds a predeterminedthreshold value. For example, the display control unit 115 f instructsthe operation screen generating unit 115 e to generate the stereoscopicimage of the message, and the operation screen generating unit 115 egenerates a stereoscopic image and sends the stereoscopic image to thedisplay control unit 115 f. Then, the display control unit 115 fdisplays the stereoscopic image in another layer different from thelayer in which the operation screen is displayed, for example, anddisplays the layer of the operation screen and the layer of thestereoscopic image in a superimposed manner. Similarly to the thumbnailicon described in the first embodiment, the operation screen generatingunit 115 e may generate a stereoscopic image by attaching a message to agroup of parallax images for messages generated in advance by performingcomputation processing so that a cuboid can be perceivedstereoscopically, for example. Alternatively, such a stereoscopic imagemay be prepared in advance for each message.

Threshold values may be provided in steps so that the stereoscopiceffect reflects the degree of importance. For example, the stereoscopiceffect may be increased as the importance of a message increases.Moreover, the color may reflect the degree of importance. For example,the message is displayed “red” for the importance of a high level,“yellow” for the importance of a middle level, and “green” for theimportance of a low level. For example, the operation screen generatingunit 115 e may store a group of parallax images for respectivestereoscopic effects and colors in advance and generate a stereoscopicimage by reading the group of parallax images and attaching a message tothe same.

A specific example will be described. For example, it is assumed thatthe medical image diagnostic apparatus 110 is in a state where it has alow available disk space for storing medical image data. Moreover, it isassumed that a message “Warning! The available disk space is low” isprepared in advance as the information for informing the operator ofsuch a state, and high importance is set to the message.

Then, for example, upon receiving a message display instruction from thereconstructing unit 115 b, the display control unit 115 f specifies thatthe importance of the message exceeds a threshold value by referring tothe importance thereof. Moreover, the display control unit 115 instructsthe operation screen generating unit 115 e to generate a stereoscopicimage of a message “Warning! The available disk space is low,” and theoperation screen generating unit 115 e generates a stereoscopic imageand sends the stereoscopic image to the display control unit 115 f.Then, the display control unit 115 f displays a stereoscopic message“Warning! The available disk space is low.” on the operation screen.

Whether the message will be displayed as a stereoscopic image may bedetermined depending on the stage where the message is notified, forexample, as well as the warning level. For example, it is assumed thatthe message “Warning! The available disk space is low.” is likely to benotified in two stages of an imaging planning stage and an imagingstage. When this message is notified in the imaging planning stage, thismessage is notified via a sound in addition to characters and a pop-upwindow. On the other hand, when this message is notified in the imagingstage, the message is not notified via a sound from consideration ofpatients. Therefore, although the display control unit 115 f does notdisplay the message as a stereoscopic image in the imaging planningstage where a notification is performed via a sound, the display controlunit 115 f displays the message as a stereoscopic image in an imagingstage where the notification is not performed via a sound.

Advantageous Effects of Second Embodiment

As described above, according to the second embodiment, since whetherthe message will be displayed as a stereoscopic image or a planar imageis determined depending on the content of the message, the operationscreen is displayed appropriately. Thus, the operator cannot miss amessage of high importance.

Other Embodiments

Several other embodiments will be described.

Hereinabove, although an example in which thumbnail icons are displayedso as to be perceived stereoscopically has been described in the firstembodiment, and an example in which a message to be notified to theoperator is displayed so as to be perceived stereoscopically has beendescribed in the second embodiment, the embodiments are not limited tothis. For example, the medical image diagnostic apparatus 110 displaysan imaging planning screen when planning imaging, and the imagingplanning screen may be controlled to be displayed as a stereoscopicimage or a planar image depending on the content of the imaging planningscreen. For example, although the medical image diagnostic apparatus 110according to a third embodiment displays a normal imaging planningscreen as a planar image, when displaying a graph (hereinafter referredto as a time schedule screen) for planning execution of various imagingprotocols as an imaging planning screen, the medical image diagnosticapparatus 110 displays the time schedule screen as a stereoscopic image.

FIG. 18 is a diagram for describing a time schedule screen according tothe third embodiment. As illustrated in FIG. 18, a graph for creating arod graph for each imaging protocol using a time axis (the axis ofimaging in FIG. 18) as the horizontal axis is displayed on the timeschedule screen. The operator can determine the type of an imagingprotocol performed in an examination, the order thereof, and the like byinputting operations on the graph. For example, “RP” illustrated in FIG.18 means “real time prep” imaging. For example, when the operator clickson “RP” on the time schedule screen and draws a rectangle at an optionalposition on the graph, it is planned that “real time prep” imaging isperformed at the time indicated by the position where the rectangle isdrawn.

Here, for example, when generating the time schedule screen within theimaging planning screen, the operation screen generating unit 115 egenerates a graph of which the horizontal axis is “axis of imageprocessing” in addition to the graph of which the horizontal axis is“axis of imaging” and generates operation screen information so thatthese graphs are displayed in a superimposed manner so as to beperceived stereoscopically as illustrated in FIG. 18. In FIG. 18,although the graph of which the horizontal axis is “axis of imageprocessing” is illustrated to be shifted for the sake of convenience,actually the graph of which the horizontal axis is “axis of imageprocessing,” for example is displayed with the sense of protrusion orthe sense of depth. The operator can designate image data to besubjected to image processing by inputting operations on the graph forimage processing. For example, when the operator draws a rectangle a onthe “axis of image processing,” it is planned that three sets of volumedata at the center among dynamic volume data collected in five sets aresubjected to image processing.

The graph of which the horizontal axis is “axis of image processing” maynot be displayed always, and may be displayed when the operator pressesa button indicating “to display “axis of image processing”,” forexample. For example, the operation screen generating unit 115 e maygenerate operation screen information for displaying as a planar imageand generate operation screen information for displaying as astereoscopic image depending on whether this button is pressed or not.

Moreover, the embodiment is not limited to the above-describedembodiments, but “axis of imaging” of which the horizontal axis is thetime axis may be prepared for each imaging protocol, and the respectiveaxes may be arranged in the vertical direction to the display surfacewith the sense of protrusion or the sense of depth. In this way, it ispossible to obtain an advantage effect that the operator can easilyperceive the “axis of imaging” for each imaging protocol.

In the above-described embodiment, although the operation screen of themedical image diagnostic apparatus 110 has been described by way of anexample, the embodiments are not limited to this. For example, the samecan be applied to the operation screen of the workstation 130 or theoperation screen of the terminal device 140. In this case, theworkstation 130 or the terminal device 140 has the functionscorresponding to the storage unit 114, the control unit 115, and therendering processing unit 116. Moreover, when the workstation 130 or theterminal device 140 acquires medical image data archived in the imagearchiving device 120 in accordance with the DICOM standard and displaysan operation screen, additional information of DICOM can be used as“medical image data information” described in the above-describedembodiments, for example.

Moreover, in the above-described embodiment, the terminal device 140 hasbeen described to display the medical images and the like acquired fromthe image archiving device 120. However, the embodiments are not limitedto this. For example, the terminal device 140 may be directly connectedto the medical image diagnostic apparatus 110.

Moreover, in the above-described embodiment, although 9-parallax imageshave been described by way of an example, the embodiments are notlimited to this, and an optional parallax number such as two parallaxesor six parallaxes may be used, for example. Moreover, all of the designof the operation screen, the number of thumbnail icons, the method ofaligning thumbnail icons, the shape of thumbnail icons, the design of astereoscopic message can be change in an optional way.

Others

Respective constituent elements of respective apparatuses illustrated inthe drawings are functionally conceptual, and physically the sameconfiguration illustrated in the drawings is not always necessary. Thatis, the specific forms of distribution and integration of the devicesare not limited to the illustrated forms, and all or a part thereof canbe functionally or physically distributed or integrated in an arbitraryunit, according to various kinds of load and the status of use.Furthermore, all or an arbitrary part of the processing functionsperformed by the respective apparatuses can be realized by a CPU and aprogram analyzed and executed by the CPU, or can be realized as hardwareby a wired logic.

Moreover, the image processing method described in the above embodimentscan be realized when an image processing program prepared in advance isexecuted by a computer such as a personal computer or a workstation. Theimage processing program may be distributed via a network such as theInternet. Moreover, this program may be executed by being recorded in acomputer-readable recording medium such as a hard disk, a flexible disk(FD), a CD-ROM, a MO, or a DVD, and read from the recording medium by acomputer.

According to the image processing system and method of at least one ofthe embodiments described herein above, it is possible to display theoperation screen appropriately.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An image processing system comprising: a displaycapable of displaying stereoscopic images by displaying a plurality ofgroups of parallax images; and a processor configured to display anoperation screen for receiving operations on medical image data on thedisplay, display a thumbnail image for each of the respective groups ofparallax images, for selecting the medical image data on the operationscreen, and control whether the thumbnail image will be displayed as astereoscopic image or a planar image depending on the content of themedical image data selected by the thumbnail image, wherein theprocessor increases a parallax angle of each respective group ofparallax images for displaying the thumbnail image in accordance with anumber of images included in the medical image data, and when thethumbnail image is displayed as a stereoscopic image, the processordisplays the thumbnail image so that stereoscopic effect of thethumbnail image increases as the number of the medical image dataincreases.
 2. The image processing system according to claim 1, whereinwhen the thumbnail image is displayed as the stereoscopic image, theprocessor uses a group of parallax images generated by performing arendering process on the 3-dimensional medical image data.
 3. The imageprocessing system according to claim 1, wherein when the thumbnail imageis displayed as the stereoscopic image, the processor uses a group ofparallax images generated by performing a rendering process on the3-dimensional medical image data.
 4. The image processing systemaccording to claim 1, wherein the processor displays an operation screenfor receiving operations of an operator on the display, displaysnotification information to be notified to the operator on the operationscreen, and controls whether the notification information will bedisplayed as a stereoscopic image or a planar image depending on thecontent of the notification information.
 5. An image processing methodexecuted by an image processing system, comprising: displaying anoperation screen for receiving operations on medical image data on adisplay capable of displaying stereoscopic images by displaying aplurality of groups of parallax images, each displaying a respectivethumbnail image for selecting the medical image data on the operationscreen, and specifying whether the thumbnail image will be displayed asa stereoscopic image or a planar image depending on the content of themedical image data selected by the thumbnail image; and increasing aparallax angle of each respective group of parallax images fordisplaying the thumbnail image in accordance with a number of imagesincluded in the medical image data, and when the thumbnail image isdisplayed as a stereoscopic image, the processor displays the thumbnailimage so that stereoscopic effect of the thumbnail image increases asthe number of the medical image data increases.
 6. The image processingmethod executed according to claim 5, further comprising: displayingnotification information to be notified to the operator on the operationscreen, and specifying whether the notification information will bedisplayed as a stereoscopic image or a planar image depending on thecontent of the notification information; and controlling so that thenotification information is displayed on the display.